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ABSTRACT 

Before selecting any measurement instrument the test 
user should always determine what it is he wants to know. A new 
instrument. The Assessment of Career Development (ACD), attempts to 
Sissess students* knowledge about jobs* One must, however, determine 
whether the content of the ACD is similar to the content of the 
career information system the students are using to learn about these 
jobs. The paper presents the method of determining whether the 
content of the ACD was valid for measuring knowledge of job 
information if the job information system utilizes the Roe 
Occupational Classification. The paper provides the test user with 
the information needed to Sissess an information system (if employing 
Roe's classification) and, also, the method of determining the 
significant proportional differences of test content and criterion 
for any comparison. (Author) 
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THE ASSESSMENT OF CAREER DEVELOPMENT: CONTENT VALIDITY 
WITH ROE'S OCCUPATIONAL CLASSIFICATION 

The American College Testing Program (ACT) published a 
new instrument in the fall of 1973. The instrument, the As- 
sessment of Career Development (ACD), was designed to be used 
with eighth through eleventh grade students for guidance as- 
sessment as part of a school *s developmental career guidance 
program. In order to determine the value of using the ACD in 
a specific school, one must judge whether local student career 
guidance needs and objectives can be appropriately assessed by 
the ACD. The purpose of this study was to determine the con- 
tent validity of the ACD subtest, "Occupational Awareness," 
for the evaluation of occupational information programs which 
use Anne Roe's (1955) Classification of Occupations. This 
study both presents the reader with: (1) a criterion for 
content validity of assessment (if the reader's career infor- 
mation program uses the Roe System) and (2) a strategy for de- 
termining content validity with other classification systems. 
Content validity is being defined as the property of a test 
to measure a certain criterion. The ACD, the Roe System of 
classification, and the method of determining content validity 
will be discussed in the following sections. 

Description of ACD 

The ACD is a guidance assessment instrument with two 
major purposes: first, to help counselors and administrators 
obtain information needed for developing effective guidance 
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programs; and second, to assess the outcomes of career guid- 
unco programs. The scales of the instrument are structured 
around throe- phases of career development: (1) occupational 
awareness (knowledge and exploration); (2) self -awareness, 
and (3) career planning and decision-making. The evaluation 
being presented deals solely with the first section, "Occupa- 
tional Awareness," This section consists of seventy-two (72) 
multiple-choice questions designed to assess students' knowl- 
edge of occupational characteristics and occupational prepara- 
tion requirements. Unit one is, in essence, an achievement 
test of the "world of work." 

Baer and Roeber (1964) state that occupational informa- 
tion is an educational essential and, also, is essential for 
educational motivation and career decision-making. The "Job 
Knowledge" section of the ACD presents to the counselor an 
assessment tool which can aid in determining student career 
knowledge and needs for occupational information. 

The information about jobs should be presented in some 
systematic method in order to facilitate student learning 
(Crites). For the purpose of example, the Roe occupational 
classification system was selected as a model system for or- 
dering occupational information and demonstrating a method 
of determining content validity with the ACD. 

Anne Roe's Classification of Occupations 

Roe (1955) devised a two-dimensional system of occupa- 
tional classification. Her system delimited the number of 
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job titles given in the Dictionary of Occupational Titles 
(DOT) to 650. After determining, through item analysis, the 
list of job titles to bo classified, each job was then cate- 
gorized using the two level code. The first number of the 
code indicated which of the eight job activity groups ("job 
clusters") the job represents. The "job clusters" and their 
code numbers are (1) Service, (2) Business Contact, (3) Or- 
ganization, (4) Technology, (5) Outdoor, (6) Science, (7) ■ 
General Cultural, and (8) Arts and Entertainment. The second 
number of the code represented the "level of function" for 
the job. The "levels of function" and their code numbers 
are: (1) Professional and managerial with independent re- 
sponsibility, (2) Professional and managerial, (3) Semi-pro- 
fessional, (4) Skilled, (5) Semi-skilled, and (6) Unskilled. ' 
All 650 job titles were given a two-dimensional code number; 
for example, a barber was classified 1:4, meaning that the 
primary activity involves service to others while the func- 
tional level is a skilled worker. Occupational information 
systems employing the Roe Classification of Occupations em- 
ploy the above described method. 

There are many occupational information systems which 
categorize jobs according to Roe's classification. The Com- 
puterized Vocational Information System (CVIS) is the most 
widely-used program using the Roe classification. The re- 
mainder of the text presents the method and results of de- 
termining the content validity of ACD Occupational Awareness 
for assessing student knowledge of occupations. The assess- 



mont is only useful for evaluating a program using the Roe 
system. 

Method 

In order to carry out the analysis, the job titles in 
the "Occupational Awareness*' section of the ACD were coded 
according to the Roe classification system. Each item in 
the Unit was examined and each job title was classified ac- 
cording to the Roe system. All of the jobs were classified, 
including the distractors, until all 211 job titles were com- 
pleted. 

This analysis of occupations was first completed on the 
job clusters to determine the proportional content of the Roe 
list of jobs for each cluster. The number of jobs from each 
of Roe's clusters found in the ACD was compared to the total 
number of jobs for that cluster on Roe's list. A Z-test was 
used to determine if the two proportions were significantly 
different from each other (Edwards). An identical analysis 
was completed for analyzing the "levels of function.*' Edwards 
(1972) provided the formulas needed for the analyses. 

Results 

Of the 650 job titles listed by Roe, ACD ''Job Knowledge" 
surveyed 221 of them. The percentage analysis is given in 
Table I. The comparison of percentage differences was cot?- 
pleted using a Z-test . The level of significance was set at 
.05. 

O 
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(INSERT TABLE I) 

« 

For the analysis of the "job clusters" there was only 
one significant percentage difference, that difference re- 
sulted from the ACD's minimal representation for "Technology" 
(cluster 4). The statistical treatment for each "job cluster" 
is demonstrated in Table II. 

(INSERT TABLE II) 
For the "levels of function" there was no significant per- 
centage differences. The statistical treatment for each "level 
of function" is demonstrated in Table III. The results indicate 
the ACD is representative of thirteen of the fourteen job cate- 
gories. 

Conclusions 

The conclusions reached from the analysis provides a ra- 
tionale for career specialists wishing to use the ACD to assess 
their career guidance program. The subsection titled "Occupa- 
tional awareness" is generally representative of the Anne Roe 
Classification of Occupations and can, therefore, be used to as- 
sess students' knowledge of occupational characteristics and oc- 
cupational preparation requirements for career information sys- 
tems employing Roe's classification. Students learning about 
specific "job clusters" and "levels of function" can be validly 
assessed for each category except "Technology." One of the 
values of systematically providing job information is that a 
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flexible assessment program can be established. Counselors, 
career coordinators, and vocational teachers may effectively 
utilize the above strategy for establishing content validit: 
for their assessment instruments. Any assessment instrument 
can be validated according to the method described. Any 
career information program attempting to use the ACD for as- 
sessment should complete this described analysis for deter- 
mining the instruments content validity for assessing that 
program. 



TABLE I 



Percentage Comparison of ACD "Job Knowledge'* with 
Anne Roe's Clas.'if ication of Occupations 



Comparison with "Job Clusters" 


Cluster number 


1 


2 


3 


4 


5 


6 


7 


8 


T 


Number of jobs listed 
by Anno Roe 


51 


19 


100 


314 


42 


78 


23 


23 


650 


Percentage of the total 
for each "job cluster" 


7.4% 


3.0% 


15.4% 


48.3% 


6.4% 


12.0% 


3.5% 


3.5% 


100 


Number of jobs listed 
in ACD "Job Knowledge" 


30 


11 


51 


54 


14 


27 


17 


17 


221 



Percentage of the total 

for each **job cluster'* 13.5% 4.9% 23.0% 24.4% 6.3% 12.2% 7.7% 7.7% 100 



B. Comparison with **Levels of Function" 



Level of Function 
Number 


1 


2 




3 


4 


5 


6 




T 


Number of jobs listed 
by Anne Roe 


55 


111 




57 


172 


147 


lOR 




650 


Percentage of the total 
for each level 


8. 


5% 17. 


1% 


8. 


8% 26. 


5% 22. 


6% 16. 


6% 


100 


Number of jobs listed 
in ACD "Job Knowledge" 


19 


53 




25 


55 


41 


28 




221 


Percentage of the total 
for each level 


8. 


6% 23. 


9% 


11 . 


3% 24. 


9% 18. 


6% 12. 


7% 


100 
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TABLE II 
"Job Cluster" Analysis 



*Job Cluster Number 


] 


2 


3 


4 


5 


6 


7 


8 


Pl 


. 35 


. 38 


.40 


. 6G 


. 51 


. 49 


.31 


. 31 


p2 


. 65 


. 62 


.60 


. 34 


.49 


. 51 


. 69 


. 69 


P 


.10 


. 04 


.19 


.36 


.06 


. 12 


.06 


.06 


Q 


.90 


.96 


.81 


.64 


.94 


.88 


.94 


,94 


nl 


7.40 


3.00 


15.40 


48.30 


6.50 


12.00 


3.50 


3.50 


n2 


13.50 


4.90 


23.00 


24.40 


6.30 


12.20 


7.70 


7.70 


S.D. 


14 


.14 


.12 


.12 


.13 


13 


. 15 


50 


z 


-2 1 4 


-1.71 


-1 .67 


2.67 


0. 15 


-0 . 15 


-2 . 53 


-2 5*^ 


Probability of 
S irn i laf i t v 




** 


** 


* 

0.3% 


44 . 0% 




■ ** 






f t T ^ 1 

Level 


of Function" 


Analysis . 








Level of Function ' 


Number 


1 


2 


3 


4 


5 


6 




pl 




.49 


.42 


.44 


.52 


. 55 


. 57 




p2 




.51 


.58 


.56 


.48 


. 45 


. 43 




P 




,085 


.20 


.10 


.26 


.21 


. 15 




Q 




.915 


.80 


.90 


.74 


.79 


.85 




nl 




8.500 


17.10 


8,80 


26.50 


22.60 


16.60 




n2 




8.600 


23.90 


11.30 


24.90 


18.60 


12.70 




S.D. 




.13 


.13 


.13 


.14 


.12 


.14 




Z 




-.15 


-1.23 


-0. 92 


.50 


.83 


1.00 




Probability of 
Similarity 




** 


** 


** 


31.0% 


21.0% 


16 . 0% 





* p < .01 
** p ^ 100% 
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